Silhouette-Extraction for interactive VR-Systems involving dynamically changing rear-projection screens

Markerless optical motion tracking is the method of choice if you don’t want to use expensive tracking hardware or invasive equipment in a VR-System. Combined with silhouette-extraction based on background subtraction realtime interaction is reachable. But most methods using this scheme are most likely unable to adapt their background model to the fast changing unpredictable images shown on a projection screen. This restriction makes them unusable for applications involving spatially immersive displays (SID) where users are interacting in front of projection screens. This paper describes an approach to solve this challenging problem. Based on a client-server architecture it makes use of frame buffer and screen timing information delivered by the presentation computer and standard digital cameras attached to the tracking computer. The system requires no special hardware; therefore it is comparatively cheap and furthermore fast and reliable. Sufficiently accurate algorithms are used, allowing usage in a real-time system.